Binary Rewriting in Software Instruction Cache

ABSTRACT

Mechanisms are provided for dynamically rewriting branch instructions in a portion of code. The mechanisms execute a branch instruction in the portion of code. The mechanisms determine if a target instruction of the branch instruction, to which the branch instruction branches, is present in an instruction cache associated with the processor. Moreover, the mechanisms directly branch execution of the portion of code to the target instruction in the instruction cache, without intervention from an instruction cache runtime system, in response to a determination that the target instruction is present in the instruction cache. In addition, the mechanisms redirect execution of the portion of code to the instruction cache runtime system in response to a determination that the target instruction cannot be determined to be present in the instruction cache.

BACKGROUND

The present application relates generally to an improved data processingapparatus and method and more specifically to mechanisms for performingbinary rewriting in software instruction caches.

Many modern computing devices utilize a multiprocessor architecture inwhich multiple processors are provided to increase the computation powerof the computing device. One example of a modern multiprocessorarchitecture is the Cell Broadband Engine (CBE) available fromInternational Business Machines Corporation or Armonk, N.Y. With theCBE, a primary control processor, referred to as the PPE, is providedalong with a plurality of controlled processors, referred to assynergistic processing elements (SPEs). Each SPE has a local memory, orlocal store, into which instructions and data are copied so that the SPEmay execute instructions in the local store on data brought into thelocation store from main memory. Thus, the local store serves as both aninstruction and data cache for the SPE. Other multiprocessorarchitectures utilize similar configurations in which the processors mayhave a local instruction cache and data cache into which data andinstructions are brought before executing on the processor or having theprocessor operate on the data.

Typically, the local store, or cache, of a modern day multiprocessorarchitecture is designed to be much smaller in storage size than themain memory. Thus, executing code larger than the processor's localstore or cache size requires a strategy for swapping pieces of code, orcode segments, into the local store or cache before use. In some cases,a code segment may include branch instructions whose target instructionis located in a different code segment that may not be currently presentin the local store or cache. This would require a strategy for bringingin the code segment corresponding to the target instruction from mainmemory.

SUMMARY

In one illustrative embodiment, a method, in a data processing system,is provided for dynamically rewriting branch instructions in a portionof code. The method comprises executing, in a processor of the dataprocessing system, a branch instruction in the portion of code. Themethod further comprises determining, by the processor, if a targetinstruction of the branch instruction, to which the branch instructionbranches, is present in an instruction cache associated with theprocessor. Moreover, the method comprises directly branching, by theprocessor, execution of the portion of code to the target instruction inthe instruction cache, without intervention from an instruction cacheruntime system, in response to a determination that the targetinstruction is present in the instruction cache. In addition, the methodcomprises redirecting, by the processor, execution of the portion ofcode to the instruction cache runtime system in response to adetermination that the target instruction cannot be determined to bepresent in the instruction cache.

In other illustrative embodiments, a computer program product comprisinga computer useable or readable medium having a computer readable programis provided. The computer readable program, when executed on a computingdevice, causes the computing device to perform various ones, andcombinations of, the operations outlined above with regard to the methodillustrative embodiment.

In yet another illustrative embodiment, a system/apparatus is provided.The system/apparatus may comprise one or more processors and a memorycoupled to the one or more processors. The memory may compriseinstructions which, when executed by the one or more processors, causethe one or more processors to perform various ones, and combinations of,the operations outlined above with regard to the method illustrativeembodiment.

These and other features and advantages of the present invention will bedescribed in, or will become apparent to those of ordinary skill in theart in view of, the following detailed description of the exampleembodiments of the present invention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The invention, as well as a preferred mode of use and further objectivesand advantages thereof, will best be understood by reference to thefollowing detailed description of illustrative embodiments when read inconjunction with the accompanying drawings, wherein:

FIG. 1 is an example embodiment of an information handling systemcapable of implementing aspects of the illustrative embodiments setforth herein;

FIG. 2 is a diagram showing embodiments of a software controlledinstruction cache (icache) manager translating a target instructionspace address to other types of instruction addresses;

FIG. 3 is a diagram showing a cache line that includes direct branchinstructions and corresponding branch descriptors in accordance with oneillustrative embodiment;

FIG. 4 is a flowchart showing steps taken in compiling various branchinstruction types in accordance with one illustrative embodiment;

FIG. 5 is a flowchart showing steps taken in executing a direct branchinstruction in a runtime environment in accordance with one illustrativeembodiment;

FIG. 6 is a flowchart showing steps taken in executing an indirectbranch instruction in a runtime environment in accordance with oneillustrative embodiment;

FIG. 7 is a flowchart showing steps taken in executing a function returninstruction in a runtime environment in accordance with one illustrativeembodiment;

FIG. 8A shows a target instruction space address translated into atarget local store address in accordance with one illustrativeembodiment;

FIG. 8B is a flowchart showing steps taken in an icache managertranslating a target instruction space address to a target instructiontag (ITAG) in accordance with one illustrative embodiment;

FIG. 8C is a flowchart showing steps taken in an icache managertranslating a target instruction space address to a target backing storeaddress in accordance with one illustrative embodiment;

FIG. 9 illustrates information handling system, which is anotherembodiment of a simplified example of a computer system capable ofperforming the computing operations described herein;

FIG. 10 is an example diagram of a branch stub in accordance with oneillustrative embodiment;

FIG. 11 is an example diagram illustrating a rewriting of a linkregister in accordance with one illustrative embodiment;

FIG. 12 is an example diagram illustrating an overall operation of oneillustrative embodiment for rewriting branch instructions using branchstubs;

FIG. 13 is a flowchart outlining an example operation of a compiler forrewriting branch instructions in accordance with one illustrativeembodiment;

FIG. 14 is a flowchart outlining an example operation or an instructioncache runtime system for performing dynamic rewriting of branchinstructions in accordance with one illustrative embodiment;

FIG. 15 is an example diagram illustrating a call stub in accordancewith one illustrative embodiment;

FIG. 16 is a flowchart outlining an example operation of an instructioncache manager when handling the eviction of a cache line in accordancewith one illustrative embodiment;

FIG. 17 is an example diagram illustrating one illustrative embodimentof a source code block listing quad word;

FIG. 18 is a flowchart outlining an example operation of a compiler whenanalyzing original code and inserting a list of possible source codeblocks targeting a current code block in accordance with oneillustrative embodiment;

FIG. 19 is a flowchart outlining an example operation of the instructioncache runtime system in accordance with one illustrative embodiment,with regard to handling eviction of a cache line; and

FIG. 20 is a flowchart outlining an example operation for performingcode optimization using a partitioned call graph in accordance with oneillustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiments provide a mechanism for statically anddynamically rewriting branch instructions so that they may be moreoptimally executed in view of the limited size of local stores or cachesof a processor. In particular, the mechanisms of the illustrativeembodiments provide functionality for rewriting code, both statically bya compiler and dynamically by an instruction cache runtime system, tothereby generate and utilize branch stubs in the code for branchinstructions. The branch stubs may be used as a basis for holdinginformation about the branch instruction for use in determining whethera jump directly to a target of the branch instruction in the instructioncache may be possible or not. Examples of branch instructions includefunction calls, conditional branch instructions, unconditional branchinstructions, function returns, and the like, as is generally known inthe art.

In addition, the mechanisms of the illustrative embodiments providefunctionality for performing dynamic rewriting of branches based onwhether or not the possible targets of the branches are known or not.The rewriting of the branch instructions allows the code to directlyjump to target code in the instruction cache when possible rather thanhaving to go through an instruction cache manager to determine if thetarget code is present in the instruction cache or not. In general, whenthe target code is loaded into the instruction cache, the branchinstruction is rewritten as a jump to the location of the target code inthe instruction cache. When the target code is evicted from theinstruction cache, the branch instruction is restored since it cannotjump to the target code directly but must trigger a software instructioncache mechanism to handle the jump to the target code.

As will be appreciated by one skilled in the art, the present inventionmay be embodied as a system, method, or computer program product.Accordingly, aspects of the present invention may take the form of anentirely hardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,aspects of the present invention may take the form of a computer programproduct embodied in any one or more computer readable medium(s) havingcomputer usable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablemedium would include the following: an electrical connection having oneor more wires, a portable computer diskette, a hard disk, a randomaccess memory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), an optical fiber, a portablecompact disc read-only memory (CDROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.In the context of this document, a computer readable storage medium maybe any tangible medium that can contain or store a program for use by orin connection with an instruction execution system, apparatus, ordevice.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, in abaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Computer code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, radio frequency (RF), etc., or anysuitable combination thereof.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java™, Smalltalk™, C++, or the like, and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer, or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to the illustrativeembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions thatimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus, or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

FIG. 1 is an example embodiment of an information handling systemcapable of implementing aspects of the illustrative embodiments setforth herein. The example information handling system shown in FIG. 1 isan example of a Cell Broadband Engine (CBE) architecture available fromInternational Business Machines Corporation of Armonk, N.Y. It should beappreciated, however, that the present invention is not limited to usewith the CBE and other information handling system architectures,configurations, and the like, may be used without departing from thespirit and scope of the illustrative embodiments.

Processor element architecture (100) includes multiple heterogeneousprocessors that share a common memory and a common bus. Processorelement architecture (PEA) 100 sends and receives information to/fromexternal devices through input output 190, and distributes theinformation to control plane 110 and data plane 140 using processorelement bus 180. Control plane 110 manages PEA 100 and distributes workto data plane 140.

Control plane 110 includes processing unit 120, which runs operatingsystem (OS) 125. For example, processing unit 120 may be a Power PC corethat is embedded in PEA 100 and OS 125 may be a Linux operating system.Processing unit 120 manages a common memory map table for PEA 100. Thememory map table corresponds to memory locations included in PEA 100,such as L2 memory 130 as well as non-private memory included in dataplane 140.

Data plane 140 includes Synergistic Processing Elements (SPE) 150, 160,and 170. Each SPE processes data information and each SPE may havedifferent instruction sets. For example, PEA 100 may be used in awireless communications system and each SPE may be responsible forseparate processing tasks, such as modulation, chip rate processing,encoding, and network interfacing. In another example, each SPE may haveidentical instruction sets and may be used in parallel to performoperations benefiting from parallel processes. Each SPE includes asynergistic processing unit (SPUs 155, 165, 175), which is a processingcore, such as a digital signal processor, a microcontroller, amicroprocessor, or a combination of these cores. Each SPU includes alocal storage area (local stores 158, 168, and 178), which may besegmented into private and non-private memory. Each of local stores 158,168, and 178 store a cache directory and tag directory (instructionand/or data), along with a cache manager (see FIG. 2 and correspondingtext for further details). And, each cache manager within each SPUmanages their corresponding cache and retrieves data from backing store135 accordingly.

While FIG. 1 shows one information handling system, an informationhandling system may take many forms. For example, an informationhandling system may take the form of a desktop, server, portable,laptop, notebook, or other form factor computer or data processingsystem. In addition, an information handling system may take other formfactors such as a personal digital assistant (PDA), a gaming device, ATMmachine, a portable telephone device, a communication device or otherdevices that include a processor and memory (see FIG. 9 andcorresponding text for another embodiment of an information handlingsystem for performing compilation functions as described herein).

FIG. 2 is a diagram showing embodiments of an instruction cache (icache)manager translating a target instruction space address to other types ofinstruction addresses. Local store 200 resides within a synergisticprocessing unit (e.g., SPU's 155, 165, or 175) and includes instructioncache (icache) manager 220, instruction tag (ITAG) directory 225, andinstruction cache 230.

In order to effectively move instructions from backing store 135 toicache 230, icache manager 220 requires an effective address withinbacking store 135 that corresponds to the location of the targetinstruction (target backing store address 280), as well as a localstorage address within icache 230 to store the instruction (target localstore address 260). In addition, icache manager 220 requires aninstruction tag that corresponds to the target instruction in order toutilize ITAG directory 225 and determine a cache hit or a cache miss(target ITAG 270). As discussed herein, icache manager 220 uses aninstruction's “instruction space address” (target instruction spaceaddress 255) to compute an instruction's target backing store address,target local store address, and target ITAG. The instruction spaceaddress is an address given to an instruction at link/load time basedupon the location of the instruction in a program (program 250).

When icache manager 220 is requested (by an application thread) tobranch to a particular target instruction, icache manager 220 determineswhether the target instruction already resides in icache 230 by checkingwhether the target instruction's corresponding instruction tag (ITAG) islocated in ITAG directory 225 (target ITAG 270). In order to generatetarget ITAG 270, icache manager 220 uses target ITAG subroutine 265,which generates target ITAG 270 using target instruction space address255 and a tag array mask (see FIG. 8B and corresponding text for furtherdetails).

When icache manager 200 needs to load the target instruction frombacking store 135 into icache 230, icache manager 220 computes thelocation of the target instruction in backing store 135 (target backingstore address 280) as well as the location to store the targetinstruction in icache 230 (target local store address 260). Icachemanager 220 utilizes target instruction space address 255 to generatethese addresses as discussed below.

In the example shown in FIG. 2, target instruction space address 255 is32 bits. Target instruction space address 255's 18 least significantbits, however, is target local store address 260 because local store 200is 256 KB in size and only requires 18 bits for addressing into icache230. As such, icache manager 220 masks off target instruction spaceaddress 255's 14 most significant bits in order to generate target localstore address 260 (see FIG. 8A and corresponding text for furtherdetails). As those skilled in the art can appreciate, more or less bitsmay be required for target local store address 260 based upon the sizeof local store 200.

In order to identify the location in backing store 135 to retrieve theinstruction (target backing store address 280), icache manager 220 usesbacking store subroutine 275, which generates an offset using targetinstruction space address 255 and an icache base address, and uses theoffset and a mask to generate target backing store address 280 (see FIG.8C and corresponding text for further details). In turn, icache manager220 retrieves the target instruction from target backing store address280; stores the target instruction in target local store address 260;and updates ITAG directory 225 accordingly.

FIG. 3 is a diagram showing a cache line that includes direct branchinstructions and corresponding branch descriptors in accordance with oneillustrative embodiment. A “direct” branch instruction is an instructionin which the address of a next instruction to execute, i.e. the targetof the branch instruction, is explicitly specified. An “indirect” branchinstruction is one in which the address of the next instruction toexecute is not explicitly specified but instead the argument of thebranch instruction specifies where the address is located. Thus, anexample of an indirect branch instruction could be of the type “jumpindirect on the r1 register,” which means that the next instruction tobe executed would be at the address whose value is stored in registerr1. It should be noted that indirect branch instructions can also dependon the value of a memory location, rather than a specified register.Thus, with indirect branch instructions, the address of the nextinstruction to execute is not known at compile time, but is only knownonce the branch instruction is executed, i.e. at runtime. With directbranch instructions, the target of the branch instruction is known atcompile time.

One aspect of the illustrative embodiment is that, during sourcecompilation, a compiler adds a “branch descriptor” into a program foreach direct branch instruction and, in turn, an icache manager loads thebranch descriptors into the same instruction cache line during runtimeexecution. FIG. 3 shows instruction cache 230, which includes cache line300. Cache line 300 includes multiple instructions, some of which aredirect branch instructions 310 and 320. Cache line 300 also includesbranch descriptors 330 and 350, which correspond to direct branchinstructions 310 and 320, respectively.

When an application thread executes direct branch instruction 310,direct branch instruction 310 branches to branch descriptor 330. Branchdescriptor 330 includes trampoline branch instruction 335 and targetinstruction space address 345. A trampoline branch is an instructionthat, when executed, branches to, or “bounces” to icache manager 220,thus causing a trampoline effect. When branch instruction 310 branchesto branch descriptor 330, trampoline branch instruction 335 is invoked,which branches to icache manager 220 and loads pointer 340 into a linkregister via a branch and set link instruction, which points back tobranch descriptor 330. In turn, icache manager 220 uses pointer 340 tolocate branch descriptor 330 and retrieve target instruction spaceaddress 345. Icache manager 220 uses target instruction space address345 to generate various addresses and locate a target instructioncorresponding to direct branch instruction 310 (see FIG. 5 andcorresponding text for further details).

Likewise, when the application thread executes direct branch instruction320, the branch instruction branches to branch descriptor 350, which isdifferent than branch descriptor 330. Branch descriptor 350 includestrampoline branch instruction 355 and target instruction space address365. Target instruction space address 365 corresponds to a location ofthe target instruction for branch instruction 320. When branchinstruction 320 branches to branch descriptor 350, trampoline branchinstruction 355 is invoked, which branches to icache manager 220 andloads pointer 360 into the link register via a branch and set linkinstruction, which points back to branch descriptor 350. In turn, icachemanager 220 uses pointer 360 to locate branch descriptor 350 andretrieve target instruction space address 365.

FIG. 4 is a flowchart showing steps taken in compiling various branchinstruction types in accordance with one illustrative embodiment. Theillustrative embodiments described herein includes embodimentsdiscussing code compilation/linking, and embodiments discussing codeexecution (e.g., runtime environment). FIG. 4 shows steps taken by acompiler (and/or linker) when the compiler encounters a branchinstruction while compiling source code.

Processing commences at 400, whereupon processing retrieves aninstruction from source code 410 at step 405. A determination is made asto whether the retrieved instruction is a branch instruction (decision420). If the instruction is not a branch instruction, decision 420branches to “No” branch 422 whereupon processing compiles theinstruction (step 425) in a traditional manner known to those skilled inthe art.

On the other hand, if the retrieved instruction is a branch instruction,decision 420 branches to “Yes” branch 428, whereupon a determination ismade as to the type of branch instruction (decision 430). If theinstruction is a direct branch instruction, decision 430 branches to“Direct” branch 432 whereupon processing generates a branch descriptorthat includes a trampoline branch instruction and a target instructionspace address at step 440. During runtime execution, the direct branchinstruction branches to the trampoline branch instruction, which sends apointer to the icache manager and, in turn, the icache manager retrievesthe target instruction space address from the branch descriptor (seeFIGS. 3, 5, and corresponding text for further details). At step 450,processing compiles the branch instruction and branch descriptor.

If the branch type is an indirect branch, however, decision 430 branchesto “Indirect” branch 438, whereupon processing generates a number ofinstructions (step 460) to handle the indirect branch during runtimeexecution. The instructions include an instruction to translate theindirect branch's target instruction space address to an instruction tag(ITAG) along with an instruction to look up the ITAG in an icachedirectory (icache directory 225 shown in FIG. 2). Processing alsogenerates instructions in order to handle hit or miss outcomes of thelookup. For example, processing generates an instruction that translatesthe target instruction space address to a target local store address inorder to branch to the instruction in icache if the lookup results in acache hit, and also generates an instruction that calls the icachemanager if the lookup results in a cache miss. At step 470, processingcompiles the instructions.

A determination is made as to whether there are more instructions toprocess in source code 410 (decision 480). If there are moreinstructions to process, processing branches to “Yes” branch 482, whichloops back to retrieve and process the next instruction. This loopingcontinues until there are no more source code instruction to process, atwhich point decision 480 branches to “No” branch 488 whereuponprocessing ends at 490.

FIG. 5 is a flowchart showing steps taken in executing a direct branchinstruction in a runtime environment in accordance with one illustrativeembodiment. In some of the illustrative embodiments described herein abranch descriptor is inserted into a cache line for each direct branchinstruction (see FIG. 4 and corresponding text for further details). Inturn, the branch descriptor includes information to pass onto an icachemanager to process the direct branch instruction.

Processing commences at 500, whereupon processing (e.g., applicationthread) loads a direct branch instruction at step 505. The direct branchinstruction was arranged at load time to branch to a trampoline branchinstruction included in the direct branch's corresponding branchdescriptor (see FIG. 3 and corresponding text for further details). Assuch, at step 510, processing branches to the trampoline branchinstruction included in the corresponding branch descriptor. Thetrampoline branch instruction, at step 515, branches to the icachemanager and sends a branch descriptor pointer to the icache manager(e.g., stores the branch descriptor pointer in an icache link register).

Icache manager processing commences at 530, whereupon the icache managerretrieves the branch descriptor pointer from the link register at step535. At step 540, the icache manager uses the branch descriptor pointerto locate and extract the target instruction space address from thebranch descriptor. The target instruction space address is an addressthat corresponds to the direct branch's target instruction.

At step 545, the icache manager computes a target ITAG from the targetinstruction space using a tag array mask (see FIG. 8B and correspondingtext for further details). Next, the icache manager looks up the targetITAG in ITAG directory 225 in order to determine whether the targetinstruction is located in icache 230 (step 550).

A determination is made as to whether ITAG directory 225 includes thetarget ITAG (decision 560). If ITAG directory 225 includes the targetITAG (cache hit), decision 560 branches to “Yes” branch 562 whereuponthe icache manager translates the target instruction space address to atarget local store address at step 564 by masking off a portion of thetarget instruction space address (see FIG. 8A and corresponding text forfurther details). Then, at step 565, the icache manager branches to thetarget instruction located at the target local store address withinicache 230; passes control back to the application thread; and ends at570. The application thread continues program execution at step 520, andends at 525.

On the other hand, if ITAG directory 225 fails to include the targetITAG (cache miss), decision 560 branches to “No” branch 568 whereuponprocessing computes a target backing store address from the targetinstruction space address using an offset and mask at step 570 (see FIG.8C and corresponding text for further details). The target backing storeaddress corresponds to the location of the target instruction stored inbacking store 135. Next, the icache manager translates the targetinstruction space address to a target local store address at step 575 bymasking off a portion of the target instruction space address (see FIG.8A and corresponding text for further details). The target local storeaddress is the location in icache 230 that the target instruction isstored. At step 580, the icache manager loads the target instructionfrom backing store 135 into icache 230. At this point, the icachemanager branches to the target instruction located at the target localstore address within icache 230 and passes control back to theapplication thread (step 565), which continues program execution at step520. Icache manager processing ends at 570 and application threadprocessing ends at 525.

FIG. 6 is a flowchart showing steps taken in executing an indirectbranch instruction in a runtime environment in accordance with oneillustrative embodiment. Indirect branch processing commences at 600,whereupon processing (e.g., application thread) computes a targetinstruction tag (ITAG) using a target instruction space address locatedin register store 615 that corresponds to the indirect branchinstruction's target instruction (step 610). The target instructionspace address was stored in register store 615 from a previousinstruction (hence an indirect branch instruction).

Processing looks up the target ITAG in ITAG directory 225 at step 620,and a determination is made as to whether the lookup results in a cachehit or a cache miss (decision 630). If a cache miss occurs, decision 630branches to “No” branch 632, whereupon processing calls icache manager220 to move the target instruction from backing store 135 into icache230 (step 640). Icache manager 220, in turn, translates the targetinstruction space address to a target backing store address and a targetlocal store address in order to move the target instruction from backingstore 135 into icache 230 (see FIGS. 5, 8A, 8C, and corresponding textfor further details). On the other hand, if a cache hit occurs, decision630 branches to “Yes” branch 638, bypassing icache manager steps.

At step 650, processing translates the target instruction space addressto the target local store address and, at step 660, processing retrievesthe target instruction from icache 230 at the target local storeaddress. Processing continues program execution at step 670, and ends at680.

FIG. 7 is a flowchart showing steps taken in executing a function returninstruction in a runtime environment in accordance with one illustrativeembodiment. Processing commences at 700, whereupon processing executes afunction return instruction and, in doing so, creates a target localstore address (icache address) as the return value, which is stored in alink register included in register store 720 (step 705).

While waiting for the instruction to finish, a determination is made asto whether the target instruction is being evicted from the icache(decision 710). If the target instruction address is being evicted fromthe icache, decision 710 branches to “Yes” branch 712, whereuponprocessing translates the target local store address to a targetinstruction space address and stores the target instruction spaceaddress in the link register at step 715. The target instruction spaceaddress is stored in the link register such that when the functionreturn completes, processing realizes that the target instruction is nolonger in icache 230 and should call the icache manager to move thetarget instruction from backing store 135 into icache 230 (discussedbelow). On the other hand, if the target instruction is not evicted fromthe icache, decision 710 branches to “No” branch 718 bypassingtranslation steps.

A determination is made as to whether the function return has completed(decision 730). If the function return has not yet completed, decision730 branches to “No” branch 732, which loops back to monitor functionreturn completion and target instruction eviction. This loopingcontinues until the function return completes, at which point decision730 branches to “Yes” branch 738 whereupon processing retrieves the linkregister value from register store 720 (step 740).

A determination is made as to whether the link register value is atarget local store address (TLSA) or a target instruction space address(TISA) (decision 750). If the link register value is a targetinstruction space address, decision 750 branches to “TISA” branch 752,whereupon processing calls icache manager 220 to move the targetinstruction from backing store 135 into icache 230 (step 760). Icachemanager 220, in turn, translates the target instruction space address toa target backing store address and a target local store address in orderto move the target instruction from backing store 135 into icache 230(see FIGS. 5, 8A, 8C, and corresponding text for further details).Processing, at step 765, converts the target instruction space addressto the target local store address. On the other hand, if the linkregister value is a target local store address (e.g., target instructionwas not evicted), decision 750 branches to “TLSA” branch 758 bypassingicache manager calling steps.

At step 770, processing branches to the target address at the targetlocal store address and continues program execution at step 780.Processing ends at 790.

FIG. 8A shows a target instruction space address translated into atarget local store address in accordance with one illustrativeembodiment. The target instruction space address discussed hereinincludes 32 bits and the target local store address discussed hereinincludes 18 bits. As those skilled in the art can appreciate, more orless bits may be used for the target instruction space address and/orthe target local store address. The target instruction space addressincludes the target local store address in its 18 least significant bits(see FIG. 2 and corresponding text for further details). As such, inorder to translate the target instruction space address to the targetlocal store address, the icache manager masks off target instructionspace address 800's 14 most significant bits using mask 810, whichleaves 18 least significant bits as target local store address 820.

FIG. 8B is a flowchart showing steps taken in an icache managertranslating a target instruction space address to a target instructiontag (ITAG) in accordance with one illustrative embodiment. Icachemanager processing commences at 830, whereupon the icache managergenerates an instruction tag index using the target instruction spaceeffective address and a tag array mask (step 840). In one embodiment, atag array is an array of structures with one element per cache line.Each element includes an instruction space address of a correspondingline that is currently resident. As such, in this embodiment:

tag array mask=Number of cache lines−1

ITAG index=(TISA/linesize)&tag mask

Next, at step 850, the icache manager generates the target instructiontag at step 850 from the generated ITAG index, such as by using theinstruction:

Target ITAG=tag[ITAG index]

The icache manager may then use the target ITAG to look in the ITAGdirectory and determine whether a corresponding target instruction islocated in the icache or whether the icache manager should retrieve thetarget instruction from backing store and load it into the icache.Processing ends at 860

FIG. 8C is a flowchart showing steps taken in an icache managertranslating a target instruction space address to a target backing storeaddress. Icache manager processing commences at 870, whereupon theicache manager generates an offset using the target instruction spaceaddress (TISA) and icache base address (step 880), such as with theinstruction:

offset=(TISA−icachebase)

Next, at step 890, the icache manager generates the target backing storeaddress (TBSA) using the generated offset and mask that preserves anindex into a cache array, such as with the instruction:

TBSA=(offset&mask that preserves idx into cache array)|(offset>>(18−log2icachesize))

where “&” is a bitwise AND operator; “|” is a bitwise OR operator; and“>>” is a bitwise right shift. The icache manager may then use thetarget backing store address to retrieve the target instruction frombacking store at the target backing store address, and load the targetinstruction into the icache (see FIGS. 5-7 and corresponding text forfurther details). Processing ends at 895.

FIG. 9 illustrates information handling system 900, which is anotherembodiment of a simplified example of a computer system capable ofperforming the computing operations described herein. Informationhandling system 900 includes one or more processors 910 coupled toprocessor interface bus 912. Processor interface bus 912 connectsprocessors 910 to Northbridge 915, which is also known as the MemoryController Hub (MCH). Northbridge 915 connects to system memory 920 andprovides a means for processor(s) 910 to access the system memory.Graphics controller 925 also connects to Northbridge 915. In oneembodiment, PCI Express bus 918 connects Northbridge 915 to graphicscontroller 925. Graphics controller 925 connects to display device 930,such as a computer monitor.

Northbridge 915 and Southbridge 935 connect to each other using bus 919.In one embodiment, the bus is a Direct Media Interface (DMI) bus thattransfers data at high speeds in each direction between Northbridge 915and Southbridge 935. In another embodiment, a Peripheral ComponentInterconnect (PCI) bus connects the Northbridge and the Southbridge.Southbridge 935, also known as the I/O Controller Hub (ICH) is a chipthat generally implements capabilities that operate at slower speedsthan the capabilities provided by the Northbridge. Southbridge 935typically provides various busses used to connect various components.These busses include, for example, PCI and PCI Express busses, an ISAbus, a System Management Bus (SMBus or SMB), and/or a Low Pin Count(LPC) bus. The LPC bus often connects low-bandwidth devices, such asboot ROM 996 and “legacy” I/O devices (using a “super I/O” chip). The“legacy” I/O devices (998) can include, for example, serial and parallelports, keyboard, mouse, and/or a floppy disk controller. The LPC busalso connects Southbridge 935 to Trusted Platform Module (TPM) 995.Other components often included in Southbridge 935 include a DirectMemory Access (DMA) controller, a Programmable Interrupt Controller(PIC), and a storage device controller, which connects Southbridge 935to nonvolatile storage device 985, such as a hard disk drive, using bus984.

ExpressCard 955 is a slot that connects hot-pluggable devices to theinformation handling system. ExpressCard 955 supports both PCI Expressand USB connectivity as it connects to Southbridge 935 using both theUniversal Serial Bus (USB) the PCI Express bus. Southbridge 935 includesUSB Controller 940 that provides USB connectivity to devices thatconnect to the USB. These devices include webcam (camera) 950, infrared(IR) receiver 948, keyboard and trackpad 944, and Bluetooth device 946,which provides for wireless personal area networks (PANs). USBController 940 also provides USB connectivity to other miscellaneous USBconnected devices 942, such as a mouse, removable nonvolatile storagedevice 945, modems, network cards, ISDN connectors, fax, printers, USBhubs, and many other types of USB connected devices. While removablenonvolatile storage device 945 is shown as a USB-connected device,removable nonvolatile storage device 945 could be connected using adifferent interface, such as a Firewire interface, etcetera.

Wireless Local Area Network (LAN) device 975 connects to Southbridge 935via the PCI or PCI Express bus 972. LAN device 975 typically implementsone of the IEEE 802.11 standards of over-the-air modulation techniquesthat all use the same protocol to wireless communicate betweeninformation handling system 900 and another computer system or device.Optical storage device 990 connects to Southbridge 935 using Serial ATA(SATA) bus 988. Serial ATA adapters and devices communicate over ahigh-speed serial link. The Serial ATA bus also connects Southbridge 935to other forms of storage devices, such as hard disk drives. Audiocircuitry 960, such as a sound card, connects to Southbridge 935 via bus958. Audio circuitry 960 also provides functionality such as audioline-in and optical digital audio in port 962, optical digital outputand headphone jack 964, internal speakers 966, and internal microphone968. Ethernet controller 970 connects to Southbridge 935 using a bus,such as the PCI or PCI Express bus. Ethernet controller 970 connectsinformation handling system 900 to a computer network, such as a LocalArea Network (LAN), the Internet, and other public and private computernetworks.

The above mechanisms provide the ability for code to jump, through theuse of a trampoline instruction, to the instruction cache manager of theinstruction cache to check whether the target code for a branchinstruction is in the instruction cache or not. If not, the instructioncache manager will bring the target code into the cache. The instructioncache manager will then cause the execution to jump to where the targetcode is present in the instruction cache. This allows code segments tobe brought from a backing store, or main memory, of the system into alocal store or instruction cache associated with a processor. Thismechanism may be replicated with each processor of a multi-processorsystem such that each processor may manage the moving of instructionsinto and out of the local stores or instruction caches associated withthe individual processors.

While this mechanism generally works well, further performanceenhancement and optimization may be achieved by providing mechanisms toavoid the overhead of redirecting execution to the instruction cachemanager, having the instruction cache manager perform its checks andoperations, and then returning to the original code. The mechanisms ofthe illustrative embodiments provide further functionality todynamically rewrite branch instructions utilizing branch stubs, i.e.small portions of code that stand in for, or replace, anotherprogramming function. The branch stubs maintain information about branchinstructions and serve as a mechanism for jumping directly to targetcode in the instruction cache when possible. The branch stub code servesa similar purpose as the trampoline code with the exception that theoverhead of the instruction cache manager is avoided when the targetcode exists within the instruction cache. In the case that the targetcode is not in the instruction cache, the execution of the code may beredirected to the instruction cache manager as previously describedabove. These mechanisms may be implemented in an information handlingsystem, such as described above with reference to FIG. 9, or other typeof information/data processing system.

The branch stubs are introduced into the code, such as at the end of acode block (a section of code grouped together, typically having a sizethat is able to fit into one or more cache lines of an instructioncache), by a compiler at compile time and may be dynamically updated atruntime by an instruction cache runtime system. The instruction cacheruntime system is the collection of software designed to support theexecution of computer programs, which may include subroutines andlibraries for common operations, implementation of programming languagecommands, type checking, debugging, code generation and optimization,and the like. In particular, the instruction cache runtime systemincludes an instruction cache manager, such as that previously describedabove, which has been modified to make use of the additional mechanismsdescribed in the further illustrative embodiments hereafter. Theoriginal branch instruction is rewritten by the compiler to branch to anaddress corresponding to the branch stub for this branch instruction andthe modified branch instruction and branch stub are included in theexecutable code so that it can be brought into the instruction cachealong with the other instructions. Again, branch instructions may befunction calls, conditional branch instructions, unconditional branchinstructions, or the like, and may be direct branch instructions orindirect branch instructions.

With direct branch instructions, i.e. branch instructions for which anaddress of a next instruction to execute is explicitly specified in thebranch instruction, the branch stubs can be used to store a branchoffset, i.e. an offset from the address of the branch instruction, thatis used to locate the original target address of the next instruction toexecute and thus, the target is explicitly identified. During runtime,after the instruction cache line corresponding to the branch instructionis fetched, the direct branch instruction can be rewritten so that ittargets the address of the actual target inside the instruction cache,rather than the address of the branch stub. In this way, furtherexecutions of the direct branch instruction will proceed directly to thetarget address of the next instruction directly without incurringinstruction cache runtime system overhead, i.e. the overhead associatedwith jumping to the instruction cache manager of the instruction cacheruntime system.

Indirect branch instructions cannot be rewritten in the same manner asdirect branch instructions using the branch stubs since the targets ofindirect branch instructions are only known at runtime and may bedifferent each time the indirect branch instructions are executed. Thus,the overhead associated with having to go to the instruction cacheruntime system can only be avoided if it can be determined that thecurrent target of the indirect branch instruction is already in theinstruction cache. The illustrative embodiments provide mechanisms fordetermining whether the target of an indirect branch instruction is inthe instruction cache or not so that a direct jump can be made using thebranch stub.

With indirect branch instructions, the instruction cache runtime systemmay monitor the execution of code to identify these indirect branchinstructions and may determine the target of the indirect branchinstruction and whether the target of the indirect branch instruction isthe same target as in a previous execution of the indirect branchinstruction. The branch stubs for indirect branch instructions, alongwith a compiler inserted check instruction, can be used to make such adetermination, as will be described in greater detail hereafter. If theindirect branch instruction is targeting the same target address as fora previous execution of the indirect branch instruction, then theexecution may jump directly to this target in the instruction cachewithout having to go through the instruction cache runtime system, e.g.,the instruction cache manager. However, if the target address of theindirect branch instruction does not match a previous target address fora previous execution of the branch instruction, then the execution mayjump to the instruction cache runtime system, such as the instructioncache manager as described above, to bring the target of the indirectbranch instruction into the instruction cache.

As mentioned above, one aspect of these illustrative embodiments is theintroduction of branch stubs for branch instructions in code provided toa compiler. The compiler may analyze the received code, identify branchinstructions in the code as well as whether the branch instruction is adirect branch instruction or an indirect branch instruction. Forexample, the compiler may identify a branch instruction as a directbranch instruction when the target of the branch instruction is adiscernable address at compile time. A branch instruction whose targetis a register or some other location whose contents are dynamicallydetermined at runtime, will be identified by the compiler as an indirectbranch instruction.

For direct and indirect branch instructions, the compiler generates abranch stub, inserts a link to the instruction cache manager into afirst portion of the branch stub, and a copy of the target address fromthe original branch instruction in a second portion of the branch stub.The original branch instruction is then rewritten to reference thebranch stub, i.e. the target of the original branch instruction ischanged to an address or offset to the branch stub for that branchinstruction to thereby generate a modified branch instruction. Forindirect branch instructions, the compiler further inserts a checkinstruction, such as prior to the branch instruction, to check whether atarget of the branch instruction matches a previous target of the samebranch instruction and to jump to one of the target instruction or theinstruction cache runtime system based on an outcome of the check. Thiswill be described in greater detail hereafter.

FIG. 10 is an example diagram of a branch stub in accordance with oneillustrative embodiment. As shown in FIG. 10, in one illustrativeembodiment, the branch stub 1000 comprises a plurality of portions ofdata 1010-1040 with each portion of data 1010-1040 storing differentinformation based on the type of branch instruction that is replaced bythe branch stub 1000. In the depicted example, the branch stub 1000 iscomprised of four words 1010-1040, with each word being a fixed sizenumber of bits of data. In general the first portion 1010 of the branchstub 1000, e.g., the first word, is used for storing an address to linkregister and a cache miss handler in the first portion 1010, which maybe the instruction cache manager, may be part of the instruction cachemanager, or may otherwise be provided by the instruction cache runtimesystem, for example, in case the target of the branch is not present inthe instruction cache when the branch stub is processed. The secondportion 1020 may store a copy of the target effective address from theoriginal branch instruction. The last two portions 1030 and 1040, e.g.,the last two words, are free to be used by the instruction cache runtimesystem to store information about runtime status of the branchinstructions that are rewritten using the branch stub 1000 based on thetype of branch instruction being replaced or rewritten.

For example, the third portion 1030 of the branch stub 1000 may be usedby the instruction cache runtime system, for direct branch instructions,to store an offset into a cache line or cache line for locating theoriginal branch instruction. The fourth portion 1040 of the branch stub1000 may be used to store a copy of the original branch instructionbefore rewriting. With this extra information in the branch stub 1000,after the instruction cache manager fetches the instruction cache linecorresponding to the branch instruction into the instruction cache, theinstruction cache manager may cause the direct branch instruction to berewritten to target the actual target address inside the instructioncache rather than the branch stub 1000. Thereafter, when the directbranch instruction is again executed, the branch instruction will jumpdirectly to the target in the instruction cache rather than having to gothrough the instruction cache manager to check to see if the target isin the instruction cache.

For indirect branch instructions, the mechanisms of the illustrativeembodiments make use of the observation that indirect branchinstructions tend to target the same target address at runtime. Thus,for indirect branch instructions, the instruction cache runtime systemuses a sub-portion of the third portion 1030 of the branch stub 1000,which is unused for direct branch instructions, to store the targetaddress for the last target called by the indirect branch instruction.This target address for the last target called by the indirect branchinstruction, stored in sub-portion of the third portion 1030 of thebranch stub 1000, may be used by the check instruction inserted by thecompiler to check to see if the previous execution of the indirectbranch instruction targeted the same target address or not as thecurrent execution of the indirect branch instruction. If so, then thetarget instruction is already present in the instruction cache (sincethe previous execution used it) and thus, the indirect branchinstruction can jump directly to the target instruction in theinstruction cache without having to go through the instruction cachemanager. If the target address for the last target called by theindirect branch instruction does not match the target of the currentexecution of the indirect branch instruction, then the execution jumpsto the instruction cache manager to perform the necessary operations forbringing the target into the instruction cache, as previously describedabove.

It should be noted that in order to handle cache line eviction properly,such as when there is an instruction cache miss and a cache line needsto be evicted to bring in another cache line, the saved target addressin the sub-portion of the third portion 1030 of the branch stub 1000 ofthe calling indirect branch instruction needs to be cleared when thetarget instruction is evicted from the instruction cache. The same istrue for direct branches such that the target address information storedin the branch stub 1000 must be cleared when the target instruction isevicted from the instruction cache. More about eviction will be providedhereafter.

In addition to the above, the illustrative embodiments further includemechanisms for rewriting branch instruction returns. Branch instructionreturn rewriting, in accordance with one illustrative embodiment, isachieved without using branch stubs but instead makes use of a rewritingof a link register. For example with a processor element architecturesuch as shown in FIG. 1, each SPE may support 128 bit registers, each ofwhich may store a quadword. Register r0 may be used as a link register,i.e. a register that stores the address to return to when a functioncompletes execution, with only word 0 of the link register typicallybeing used. In general, when a function returns, the address stored inthe link register is loaded from the call stack and then a returnhandler, which may be part of the instruction cache manager, orotherwise provided in the instruction cache runtime system, is calledwith the link register address as input. The return handler would thencheck whether the code block to return to is still in the instructioncache or not, and if not, it will fetch the code block from the backingstore or main memory and put it into the instruction cache.

FIG. 11 is an example diagram illustrating a rewriting of a linkregister in accordance with one illustrative embodiment. As shown inFIG. 11, the mechanisms of the illustrative embodiments rewrite the linkregister 1100 so that additional information is stored in the linkregister 1100. That is, typically the link register 1100 only stores thereturn address, referred to as the load/store return address, in word 01110 of the link register 1100. However, with the mechanisms of theillustrative embodiments, the instruction cache runtime system rewritesthe link register 1100 to store a target address in word 0 1110, whichis to be used as a basis for an indirect branch instruction to thetarget address, and an effective address (EA) for the return address inword 1 1120.

With this rewriting of the link register 1100, when a branch returns,the instruction cache runtime system can perform a load of the linkregister from the stack and perform an indirect branch to the linkregister. The target of the indirect branch instruction to the linkregister will be word 0 of the link register. This indirect branchinstruction can then be handled using the mechanisms noted above withregard to indirect branches. As a result, if the target address storedin word 0 of the link register corresponds to the same target address asa previous execution of the indirect branch, as determined from a branchstub associated with the indirect branch instruction, then the executionof the code can bypass the return handler of the instruction cacheruntime system and jump directly to the target instruction in theinstruction cache. If the target addresses do not match, then theinstruction cache manager may be invoked as discussed above.

In particular, after instruction cache miss handling, which may occur asa result of a target of the branch instruction, e.g., a called function,not being present in the instruction cache, the return handler of theinstruction cache runtime system, e.g., in the instruction cachemanager, stores the instruction cache address of the code block toreturn to in the first slot or word, e.g., word 0 1110, of the linkregister 1100. The return handler also stores the correspondingeffective address of the instruction cache address of the code block toreturn to in a second slot or word, e.g., word 1 1120, of the linkregister 1100. Thus, after rewriting the link register, and assumingthat the target block of code is not evicted, when the branch returns,it will branch to the first slot or word, e.g., word 0 1110, of the linkregister 1100 directly without invoking the return handler of theinstruction cache runtime system.

It should be noted that cache eviction needs to be handled carefully inthis case. To correct return rewriting when the target block of code tobe returned to has been evicted from the instruction cache, whenever acache line is evicted, the instruction cache manager may traverse allcall stack frames through a back chain. For each call stack frame, theinstruction cache manager may examine the second slot or word, e.g.,word 1 1120, of the link register 1100. If the effective address in thesecond slot or word matches the cache line evicted, then the instructioncache manager may modify the first slot or word, e.g., word 0 1110, ofthe link register 1100 to point to the address of the return handler ofthe instruction cache runtime system. Thus, later, when the branchinstruction returns, the return handler will be invoked instead, and thereturn handler knows which cache line to fetch based on the second slotor word, e.g., word 1 1120, of the link register 1100.

FIG. 12 is an example diagram illustrating an overall operation of oneillustrative embodiment for rewriting branch instructions using branchstubs. As shown in FIG. 12, original code 1210 is provided and mayinclude one or more direct branch instructions 1212 and/or one or moreindirect branch instructions 1214. The compiler 1220 may receive theoriginal code 1210 and, as part of its optimizations, may rewrite thebranches in the original code 1210 using branch stubs with the branchinstructions referencing the branch stubs. In order to achieve thisrewriting, the compiler 1220 may be provided with branch rewriting logic1222 that performs the static analysis of the original code 1210 asdescribed previously to identify branches in the original code 1210 andrewrite these branches accordingly.

For example, the branch rewriting logic 1222 may identify the branchesin the original code 1210, generate a branch stub, e.g., branch stubs1238 and 1239, for each of the branches in the original code 1210, andpopulate the branch stubs with an address for the instruction cachemanager (for use in the case of a target block not being present in theinstruction cache), as well as an original target address from theoriginal branch instruction. The original branch instructions are thenrewritten to replace the reference to the target instruction with areference to the branch stub corresponding to the branch instruction.This is done regardless of whether the branch instruction is a directbranch instruction or an indirect branch instruction. As a result, therewritten code 1230 includes rewritten branch instructions 1232 and 1236as well as branch stubs 1238 and 1239 with a linking between therewritten branch instructions and their corresponding branch stubs(represented by the arrows in FIG. 12).

The branch rewriting logic 1222 further determines which branchinstructions in the original code 1210 are direct branch instructions1212 or indirect branch instructions 1214. For indirect branchinstructions 1214, the branch rewriting logic 1222 inserts check targetinstructions, such as check target instruction 1234. As discussed above,these check target instructions 1234, at runtime, cause a check to beperformed of the target address of the current execution of an indirectbranch instruction with a target address of a last previous execution ofthe indirect branch instruction to see if there is a match between thetwo. If there is a match, then the execution jumps to the target addressdirectly without having to go to the instruction cache (icache) runtimesystem 1240. If there is not a match, then the execution goes to theicache runtime system 1240 so that an instruction cache manager 1242 mayperform the necessary operations to ensure that the block of codetargeted by the indirect branch instruction is brought into theinstruction cache (icache) 1250.

The rewritten code 1230 may be provided to a linker (not shown) whichthen links the code with library functions and the like, and generatesexecutable code that is provided to processor 1280. The processor 1280may execute the rewritten code 1230 and, in the process execute therewritten branch instructions 1232 and 1236 using the correspondingbranch stubs 1238 and 1239. As part of the execution of theseinstructions, the icache runtime system 1240 may monitor the executionto determine when the branch instructions need to be dynamicallyrewritten. For example, with direct branch instructions, such as directbranch instruction 1232, the dynamic branch rewriting logic 1244 of theicache runtime system 1240 rewrites the direct branch instruction 1232dynamically to directly target the location 1254 within the instructioncache 1250 where the target of the original branch instruction iscurrently located, rather than pointing to the branch stub 1238. Thisdynamic rewriting is performed to the direct branch instruction in thedirect branch cache line 1252 in the icache 1250.

With indirect branch instructions, such as indirect branch instruction1236, the dynamic branch rewriting logic 1244 monitors whether thecorresponding check target instruction 1234 results in the targetaddresses matching or not. If the target addresses match, then theprocessor may access the indirect branch target cache line 1258 directlywithout having to go through the instruction cache manager 1242 of theicache runtime system 1240. If the target addresses do not match, thenthe execution branches to the instruction cache manager 1242 of theinstruction cache runtime system 1240, as specified by the first portionof the branch stub. In addition, the dynamic branch rewriting logic 1244rewrites the indirect branch stub 1239, in the indirect branch cacheline 1256, with the current target address of the current execution ofthe indirect branch instruction 1236, such as in a sub-portion of thethird word 1030 in FIG. 10. In this way, the target address stored inthe indirect branch stub in the indirect branch cache line 1256 isupdated to store the last previous target address of the indirect branchinstruction.

In addition to the above, the branch return rewriting logic 1246 of theicache runtime system 1240 operates to rewrite the link register 1260 sothat it stores the target address and effective address of the targetaddress in the link register as discussed above with regard to FIG. 11.When a branch instruction returns, it returns to the target address inthe link register using an indirect branch instruction that is handledin the same manner as other indirect branch instructions discussedabove. In addition, the instruction cache manager 1242 may perform theoperations noted above using the call stack 1270 to traverse call stackframes to determine if there is a match to the effective address of thelink register 1260 and if so, cause the branch return rewriting logic torewrite the target address in the link register 1260 to point to theinstruction cache manager 1242.

FIG. 13 is a flowchart outlining an example operation of a compiler forrewriting branch instructions in accordance with one illustrativeembodiment. As shown in FIG. 13, the operation starts with the compileranalyzing the received code (step 1305) and determining if a branchinstruction is encountered (step 1310). If a branch instruction is notencountered, the operation determines if an end of the code isencountered (step 1315). If not, the operation returns to step 1305.Otherwise the optimized code is output to the linker for generation ofexecutable code (step 1320) and the operation terminates.

If the compiler encounters a branch instruction (step 1310), thecompiler generates a branch stub for the branch instruction includingstoring the address to the instruction cache manager and the originaltarget address of the branch instruction as described above (step 1325).The compiler rewrites the branch instruction to target the branch stub(step 1330) and determines if the branch instruction is an indirectbranch instruction (step 1335). If the branch is an indirect branchinstruction, the compiler inserts a check target instruction prior tothe branch instruction that checks the target of the indirect branchinstruction against a last previous target address stored in the branchstub and jumps directly to the target if there is a match or jumps tothe instruction cache manager if there is not a match (step 1340).Thereafter, or if the branch instruction is not an indirect branchinstruction, the operation continues to step 1315.

FIG. 14 is a flowchart outlining an example operation or an instructioncache runtime system for performing dynamic rewriting of branchinstructions in accordance with one illustrative embodiment. As shown inFIG. 14, the operation starts by receiving a cache line requestrequesting an access to a particular portion of code in a cache line ofthe instruction cache (step 1405). A determination is made as to whetherthere is a cache hit or not, i.e. the cache line corresponding to thecache line request is already present in the instruction cache (step1410). If there is a cache hit, then the cache line is accessed from theinstruction cache (step 1415) and the operation terminates. If there isnot a cache hit, i.e. there is a cache miss, then a determination ismade as to whether the instruction cache has available unused space tostore a cache line corresponding to the cache line request (step 1420).If there is available space, then the requested cache line is fetchedfrom the backing store and loaded into the instruction cache (step1425), with the operation returning to step 405 where the cache linerequest is again executed.

If there is not available space in the instruction cache, an existingcache line is evicted (step 1430) and a determination is made as towhether dynamic branch rewriting is to be performed (step 1435). Ifdynamic branch rewriting is not to be performed, the operation continuesto step 1425. The determination as to whether to perform dynamic branchrewriting may include, for example, determining if a branch instructionloaded with the cache line targeted by the cache line request is adirect branch or an indirect branch. If the branch is a direct branch,then the branch instruction can be dynamically rewritten to directlyreference the location of the original target instruction in theinstruction cache rather than referencing the branch stub of the directbranch. If the branch is an indirect branch, then a check instructionmay be executed to determine if the target of the indirect branch is thesame as a last previous target as stored in the branch stub. If there isa match, then the branch instruction can jump directly to the target inthe instruction cache. If there is not a match, then the execution maybranch to the instruction branch manager and the current target may bestored in the branch stub in replacement of the last previous targetaddress.

If dynamic branch rewriting is to be performed, then the dynamic branchrewriting is performed as discussed above (step 1440). A return rewriteoperation may then be performed for the evicted cache line based on theeffective address of the target stored in the link register, aspreviously described above (step 1445). The link register may then bedynamically rewritten (step 1450). As noted above, the link register maybe rewritten to include the target address and the effective address ofthe target. An indirect branch instruction to the target address storedin the link register may be inserted at the branch return (step 1455).The operation then terminates.

Thus, the illustrative embodiments provide a mechanism for using abranch stub and runtime logic to bypass the instruction cache manager ininstances where it can be determined that the target of the branchinstruction is already present in the instruction cache. Moreover, themechanisms of the illustrative embodiments provide a functionality forrewriting a link register so that upon a return from executing a branchinstruction, an indirect branch to a target specified in the linkregister, or the instruction cache manager is made possible.

These mechanisms improve the performance of a software managedinstruction cache by rewriting a jump to the software instruction cacheruntime system as a direct jump to the target location in the softwarecache. However, when a cache line or block is evicted from theinstruction cache, all jumps directly going to this cache line or cacheline need to be restored to their original targets so that, ifnecessary, the jumps can be made to the instruction cache runtime systemin the event that the target is not in the instruction cache. This isbecause, if these jumps to the targets are not restored, the jumps maybe made to a wrong target since the correct target is no longer in theinstruction cache.

The unchaining process, i.e. retracing the jump instructions from onecache line to another in the instruction cache to identify source cachelines, i.e. the cache line from which a branch or call to a cache lineof interest, or target cache line, is especially challenging when thesource cache lines are only known at runtime, such as with indirectbranch instructions or function calls. One mechanism for handling thissituation is to flush the entire instruction cache or utilize expensivebookkeeping and table lookup algorithms which result in significantoverhead and lose the opportunity for reuse of the cache line, i.e. ifthe instruction cache is flushed and all cache lines are evicted, thenthe cache lines that were brought into the instruction cache cannot beused again.

The illustrative embodiments provide additional mechanisms for handlingcache eviction in which branch instructions have been rewritten todirectly target cache lines that are to be evicted. With thesemechanisms, the possible source cache lines of a cache line of interest,hereafter referred to as the target cache line, are dynamicallymaintained in a doubly linked list. When the target cache line is to beevicted, the linked list is scanned to restore the chained jump branchinstructions. In this way, the mechanisms of the illustrativeembodiments support efficient eviction via a doubly linked list of crossinter-cache line branches who's branch targets are in a particular cacheline, and a singly linked list of inter-cache line branches within aparticular cache line that have been put into the doubly linked lists ofthe other cache lines. Moreover, the mechanisms of the illustrativeembodiments support a partial stack scan with the stack back trace scanbeing stopped at a first match to a “victim” cache line, as describedhereafter.

The rewriting of direct function calls is very similar to rewritingbranches inside a function. However, unlike in the case of branchesinside a function, the compiler generally does not have a complete viewof all functions inside a program and thus, the compiler does not knowthe exact set of possible callers to a callee function. As a result, thecompiler cannot pre-allocate space to record all call sites, i.e. thebranch instruction that makes the function call which is also part ofthe source cache line, as in the case of branches inside a function.

The mechanisms of the illustrative embodiments address these problems byusing a linked list based solution that is able to handle an arbitrarynumber of call sites, i.e. branch instructions that branch to, or“call,” a portion of code corresponding to a target cache line, i.e. acallee function. For each callee function, i.e. each function called bya caller branch instruction, a linked list is dynamically constructedthat records rewritten call sites that call the callee function.

The linked list node is constructed as part of an extended branch stub,which is referred to herein as a “call stub,” which extends the branchstub described above with an additional quadword for tracking callsites. For example, in a first quadword, information corresponding tothe branch stub previously described above is stored. In a secondquadword, a linked list previous entry pointer and a linked list nextentry pointer for pointing to entries in a linked list of call sites arestored. The linked list previous entry pointer points to a previousentry in the linked list and the linked list next entry pointer pointsto a next entry in the linked list. This provides a doubly linked listof call sites that facilitates a more simplified deletion process fordeleting call sites from the linked list.

FIG. 15 is an example diagram illustrating a call stub in accordancewith one illustrative embodiment. As shown in FIG. 15, the call stub1500 includes a first portion 1510, which may be a first quad word, thatstores data similar to that of the branch stub previously describedabove. That is, the first portion 1510 stores, in a first word 1512, alink to the instruction cache manager in the case of a instruction cachemiss. In a second word 1514, the target address for the original branchinstruction or function call is stored. In a third word 1516, an offsetto the original branch or call instruction is stored. In a fourth word1518, a copy of the original branch or call instruction is stored.

In addition, the call stub includes a second portion 1520, which may bea second quadword, for storing linked list pointers for purposes ofhandling instruction cache eviction. As shown in FIG. 15, a first word1522 of the second portion 1520 may store a linked list previous pointerthat points to, i.e. stores an address for, a previous call stub in thelinked list. A second word 1524 of the second portion 1520 may store alinked list next pointer that points to, i.e. stores an address for, anext call stub in the linked list. The third and fourth words 1526 and1528 may be kept free and unused.

During runtime, if a cache line is evicted, the instruction cachemanager determines whether the evicted cache line is the entry of afunction. This determination may be done based on a tag placed on thecache directory when the cache line is loaded due to a function call ora separate table of function entry cache lines may be maintained, forexample. Other ways of determining whether a cache line is an entry to afunction may be used without departing from the spirit and scope of theillustrative embodiments.

If the cache line is the entry of a function, the cache line hasadditional information, before or after the actual code, that recordsthe start of the doubly linked list of call sites. This additionalinformation may be generated by the compiler and may be maintained bythe instruction cache runtime system. The instruction cache manager mayprocess the doubly linked list and each call site is “fixed” byrestoring the target of the original call or branch instruction to pointto the original target address rather than the call stub. This can beachieved by replacing the branch or call instruction with the copy ofthe original branch or call instruction stored in the fourth word 1518of the first quadword 1510 in the call stub 1500 or the fourth word 1040in the branch stub 1000 in FIG. 10. Since the information for performingthis fix is stored in the call stub 1500 or branch stub 1000 this fix ismade simple to implement by changing the binary code in the instructioncache to replace the branch instruction or call instruction.

In addition, the instruction cache manager may perform additionalfunctionality for maintaining the doubly linked call site list. Whenevera cache line is evicted, the instruction cache manager processes thedoubly linked call site list for the evicted cache line and checkswhether any of the call sites is in the doubly linked call site list. Ifso, the call site has to be removed from the linked list otherwise,after eviction of the cache line, the doubly linked call site list maybecome corrupted. For example, the next pointer in the previous callstub maybe changed to point to the next pointer in the current call stuband the previous pointer in the next call stub may be changed to pointto the previous call stub of the current call stub, thereby eliminatingthe current call stub from the linked list. Using this doubly linkedcall site list mechanism, an arbitrary number of call sites calling acallee function can be handled with undoing rewriting of call or branchinstructions when the callee function is evicted.

FIG. 16 is a flowchart outlining an example operation of an instructioncache manager when handling the eviction of a cache line in accordancewith one illustrative embodiment. As shown in FIG. 16, the operationstarts with the eviction of a cache line (step 1610). A determination ismade as to whether the cache line is an entry to a function (step 1620).If so, then a doubly linked call site list associated with the functionis processed using the call stub of the function and any other callstubs linked to directly or indirectly by the call stub (step 1630).Call sites in the doubly linked call site list are fixed by restoringthe target address from the call stubs corresponding to the call sites(step 1640). In addition, each call site inside the evicted block ischecked (step 1650) to determine if the call site is present in thedoubly linked call site list as the doubly linked call site list isprocessed (step 1660). If the call site is in the doubly linked callsite list, then the call site is removed from the doubly linked callsite list (step 1670). The operation then terminates.

Thus, in addition to providing mechanisms for rewriting branchinstructions using branch stubs, the mechanisms of the illustrativeembodiments may extend these branch stubs to call stubs that includepointers for a doubly linked list. These pointers may be used tomaintain a doubly linked call site list that can be used during cacheline eviction to determine which call sites need to be “fixed” byrestoring the target addresses originally referenced in the call sitebranch or call instructions. Mechanisms for maintaining the doublylinked call site list are also provided. These mechanisms make evictionof cache lines with rewritten call site branch instructions or callinstructions possible without corrupting the rewritten code.

To assist in handling the eviction of a cache line from the instructioncache, in addition to the mechanisms noted above, the illustrativeembodiments may, for direct branches or calls to a target code block,where each code block is stored in its own cache line, further record alist of source code blocks that branch to or call the code in the targetcode block. Since the compiler has a complete view of thefunctions/procedures in the code, for each code block, the compilerknows the exact list of source code blocks that could directly branch tothis target code block at runtime. The compiler just needs to recordthis information in the target code block so that it can be used by theinstruction cache runtime system. With the mechanisms of theillustrative embodiments, the information regarding the list of sourcecode blocks is recorded at the end of the target code block after all ofthe created branch stubs and call stubs.

To store this list of source code blocks information, the last quad wordof the current code block is reserved. FIG. 17 is an example diagramillustrating one illustrative embodiment of a source code block listingquad word. A first word 1710 of this quad word 1700 contains the numberof source code blocks that could branch to the current code block atruntime. A second word 1720 of the quad word contains the number ofbranch stubs this current code block has. A third word 1730 of the quadword contains a number of call stubs this current code block has. Afourth word 1740 of the quad word contains the start address of thebranch stubs and call stubs in the current code block. The locations,i.e. addresses, of the source code blocks 1750 are laid out just beforethe last quad word so that the instruction cache runtime system knowswhere to find them at runtime.

At runtime, when a cache line has to be evicted, the instruction cachemanager needs to fix all the rewritten source code blocks with a targetcode block that is in the evicted cache line. To achieve this, theinstruction cache manager in the instruction cache runtime system firstfetches the last quad word of the evicted cache line, which will also bethe last quadword of the code block since each code block is loaded intoits own cache line. From the first word 1710 of the quad word 1700, theinstruction cache manager knows how many source code block address havebeen encoded in the evicted cache line and where the list of addressesstart. The instruction cache manager processes the potential source codeblocks one by one. For each source code block, the instruction cachemanager locates the source code block based on the source code blockaddress encoded in the evicted cache line.

The instruction cache manager fetches the last quad word of that sourceblock and examines the second word 1720 and fourth word 1740 of the quadword 1700. As previously mentioned above, the second word 1720 recordsthe number of branch stubs in the source code block. The fourth word1740 records the start address of the branch stubs. With thisinformation, the instruction cache manager may examine each branch stuband check whether the branch stub is the one branching to the evictedcache line. Eventually, the instruction cache manager will find thebranch stub that branches to the evicted cache line.

To fix the rewritten branch instruction that branches to the evictedcache line, the instruction cache manager copies the fourth word of thebranch stub, i.e. word 1040 in FIG. 10, and replaces the rewrittenbranch instruction with the copy of the original branch instruction(prior to rewriting) stored in the fourth word 1040. As a result, thebranch instruction is restored and does not directly branch to theevicted cache line, thereby eliminating the possibility of a directbranch to a wrong target block. To the contrary, the restored branchinstruction will operate in the same manner as discussed above withregard to branching to the instruction cache manager to determine if thetarget block is in the instruction cache or not and bringing therequired block of code into the instruction cache if it is not alreadypresent in the instruction cache.

FIG. 18 is a flowchart outlining an example operation of a compiler whenanalyzing original code and inserting a list of possible source codeblocks targeting a current code block in accordance with oneillustrative embodiment. As shown in FIG. 18, for a next block of codein received original code, the compiler identifies possible sourceblocks that may target the next block of code (step 1810). The addressesof the possible source code blocks are stored in a portion of thecurrent code block after any branch stubs and/or call stubs created bythe compiler (step 1820). The compiler stores a number of the sourcecode blocks in a last portion of the current code block, e.g., the lastquad word of the current code block (step 1830). The compiler furtherstores a number of call stubs in the last portion of the current codeblock (step 1840). The compiler further stores the start address ofbranch and call stubs in the current code block in this last portion ofthe current code block (step 1850). A determination is made as towhether this is the last code block to process or not (step 1860). Ifnot, the operation returns to step 1810. Otherwise, the operationterminates.

FIG. 19 is a flowchart outlining an example operation of the instructioncache manager in accordance with one illustrative embodiment, withregard to handling eviction of a cache line with direct branches to theevicted cache line. As shown in FIG. 19, the operation starts withdetermining that a cache line needs to be evicted (step 1910). Theinstruction cache manager fetches the last quad word of the evictedcache line (step 1920). From the first word of the quad word, theinstruction cache manager determines how many source code block addresshave been encoded in the evicted cache line and where the list ofaddresses start (step 1930). The instruction cache manager processes anext potential source code block (step 1940). For the next source codeblock, the instruction cache manager locates the source code block basedon the source code block address encoded in the evicted cache line (step1950).

The instruction cache manager fetches the last quad word of that sourceblock and examines the number of branch stubs and start address of thebranch stubs in the last quad word (step 1960). The instruction cachemanager examines each branch stub and checks whether the branch stubbranches to the evicted cache line (step 1970). The instruction cachemanager then fixes the rewritten branch instruction corresponding to abranch stub that branches to the evicted cache line, by copying the copyof the original branch instruction from the branch stub and replacingthe rewritten branch instruction with the copy of the original branchinstruction (step 1980). The operation then terminates.

With the direct-mapped software instruction cache mechanisms describedabove, conflict misses sometimes are a problem to the performance of thesystem. A conflict miss is a cache miss due to the replacement policyused by the software cache. For example, in direct-mapped caches, a codeblock can be only put, or “mapped,” into one predefined cache line.Obviously, some code blocks may be mapped into the same cache line. Ifthey are used repeated, only one of them can be kept in the cache evenif there are many other unused cache lines. As a result, the cachemanager has to load and evict these cache blocks repeatedly. Such cachemisses are referred to as conflict misses.

To reduce the possibility of conflict misses, the illustrativeembodiments may further provide a compiler mechanism that is able tocollect and process all the objects, e.g., object files, or .o files,that together make up the executable code of a program, compiled by thecompiler that utilize direct target jump optimizations such as thosedescribed above. The compiler builds a whole program call graph thatlays out the objects and which objects call which other objects, as isgenerally known in the art. The compiler then partitions the call graphto minimize inter-partition transitions based on the instruction cachesize, the size of each function, and the affinity among functions.

In order to achieve such partitioning, the illustrative embodimentsweights the call graph by weighting the nodes in the call graphaccording to code size, and weights the edges of the call graphaccording to a number of calls between the particular objects at theends of the edge. A maximally weighted edge in unprocessed edges of thecall graph is selected and a determination is made as to whether theweights of the nodes associated with the selected edge are less than acache line size. If so, then the nodes are merged into a new node withthe weights being updated appropriately. If an edge having a maximumweight among unprocessed edges cannot be found, any remaining proceduresare places in one node together in one cache line. In this way, thecompiler tries to put high affinity functions into the same partition toreduce conflict misses.

Using such call graph partitioning and attempting to put high affinityfunctions into the same partition helps to reduce cache conflict missesfor the following reasons. Under call graph partitioning, one“compilation unit” is created for each call graph partition. That is,after the call graph is partitioned into several sub-graphs, each of thesub-graphs, along with its procedures, is associated with a compilationunit for the compiler to operate on. The compiler takes one compilationunit, or partition, as input and lays out the code in it starting at thebeginning of the instruction cache address space. Thus, the functionsinside one call graph partition normally will not evict each other, aslong as the code size for this call graph partition is smaller than thetotal instruction cache size.

With this call graph partitioning mechanism, the link-time optimizerfirst builds a global call graph in a manner generally known in the art.Each node in the global call graph is weighted according to thecorresponding size of the code represented by the node in the globalcall graph. Each call graph edge is weighted according to a number ofcalls between the nodes associated with the edge. This information mayall be determined statically by the compiler at compile time byanalyzing the original code and determining the size of the portions ofcode in the compiled code that correspond to the nodes in the globalcall graph and determining an estimate of the number of callsanticipated between nodes, such as based on iterations in loopsreferencing portions of code, or the like. Thus, the weights may bedetermined through static program analysis or by profiling, for example,and the result may be a weighted call graph that may be the basis forthe partitioning mechanisms. Of course other weightings may be usedwithout departing from the spirit and scope of the illustrativeembodiments.

FIG. 20 is a flowchart outlining an example operation for performingcode optimization using a partitioned call graph in accordance with oneillustrative embodiment. As shown in FIG. 20, the operation starts bybuilding a weighted call graph (step 2010). As noted above, the buildingof the weighted call graph starts by building a global call graph andthen applying weights to the nodes and edges of the global call graph.The weights of nodes may be determined according to the code size, suchas in bytes, associated with the nodes. The weights of edges may bedetermined based on an estimate of the number of calls between nodes ofthe particular edge. These weights are applied to the nodes and edges ofthe global call graph to thereby generate a weighted call graph.

A maximally weighted edge, in unprocessed edges of the call graph, isselected (step 2020). Assume for purposes of description that theselected edge is e and the nodes of the selected edge are v1 and v2. Adetermination is made as to whether such a maximally weighted edge canbe found or not (step 2030). If not, any remaining procedures/functionsare placed in the same node together in one cache line (step 2040) andthe operation ends.

If a maximally weighted edge is found, a determination is made as towhether a sum of the weights of the nodes associated with the selectededge, e.g., weight(v1)+weight(v2), are less than or equal to a cacheline size, e.g., a 1K cache line size or the like (step 2050). If so,then the nodes are merged into a new node with the weights being updatedappropriately (step 2060). The merging of nodes into a new node placesthe binary code for the nodes together in the resulting binary code thatis loaded into the instruction cache.

The weights of the new node may be set, for example, by setting theweight of the new node to the sum of the weights of the merged nodes,e.g., weight(vnew)=weight(v1)+weight(v2). The weight of the edgesbetween this new node and other nodes previously connected to the nodesof the selected edge are set, for example, equal to the sum of theweights of the edges between these other nodes and the nodes of theselected edges, e.g., weight (edge(edge(vnew,v))=weight(edge(v1,v))+weight(edge(v2, v)) for all v. Of course other weightings may beused without departing from the spirit and scope of the illustrativeembodiments. The selected edge e, i.e. edge (v1, v2), is then eliminatedfrom the weighted call graph (step 2070) and the operation returns tostep 2020 to select a next maximally weighted edge from the remainingunprocessed edges. If the sum of the weights of the nodes of theselected edge is greater than a cache line size, then the edge is markedas having been processed (step 2080) and the operation returns to step2020.

The result is a partitioned call graph in which nodes are merged intosizes corresponding to the cache line size of the instruction cachebased on their affinity. The resulting code is configured according tothis partitioned call graph such that when the code is brought into theinstruction cache the code fits within the cache lines of theinstruction cache will minimal cross-cache line references. This greatlyreduces conflict misses because of a cache line not being present in theinstruction cache.

Thus, the illustrative embodiments provide mechanisms for directlyjumping to branch targets in an instruction cache when possible throughcompiler implemented and runtime implemented rewriting of branchinstructions and function calls. The illustrative embodiments furtherprovide mechanisms for handling eviction of cache lines that correspondto rewritten branch instructions and function calls so as to avoid anyjumps to incorrect targets in the instruction cache. Further, theillustrative embodiments provide mechanisms for reducing instructioncache conflict misses by partitioning the code based on instructioncache size, function size, and affinity amongst the functions. All ofthese mechanisms work together to improve the performance of a system'sexecution of the code by reducing latency encountered by the overhead ofhaving to redirect execution to an instruction cache manager for everybranch or function call.

As noted above, it should be appreciated that the illustrativeembodiments may take the form of an entirely hardware embodiment, anentirely software embodiment or an embodiment containing both hardwareand software elements. In one example embodiment, the mechanisms of theillustrative embodiments are implemented in software or program code,which includes but is not limited to firmware, resident software,microcode, etc.

A data processing system suitable for storing and/or executing programcode will include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution.

Input/output or I/O devices (including but not limited to keyboards,displays, pointing devices, etc.) can be coupled to the system eitherdirectly or through intervening I/O controllers. Network adapters mayalso be coupled to the system to enable the data processing system tobecome coupled to other data processing systems or remote printers orstorage devices through intervening private or public networks. Modems,cable modems and Ethernet cards are just a few of the currentlyavailable types of network adapters.

The description of the present invention has been presented for purposesof illustration and description, and is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention, the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A method, in a data processing system, for dynamically rewritingbranch instructions in a portion of code, comprising: executing, in aprocessor of the data processing system, a branch instruction in theportion of code; determining, by the processor, if a target instructionof the branch instruction, to which the branch instruction branches, ispresent in an instruction cache associated with the processor; directlybranching, by the processor, execution of the portion of code to thetarget instruction in the instruction cache, without intervention froman instruction cache runtime system, in response to a determination thatthe target instruction is present in the instruction cache; andredirecting, by the processor, execution of the portion of code to theinstruction cache runtime system in response to a determination that thetarget instruction cannot be determined to be present in the instructioncache.
 2. The method of claim 1, wherein the branch instruction is arewritten branch instruction that has been rewritten by a compiler froman original branch instruction having an original target address, to arewritten branch instruction with an associated branch stub.
 3. Themethod of claim 2, wherein the branch stub stores a link to aninstruction cache manager of the instruction cache runtime system andthe original target address of the original branch instruction.
 4. Themethod of claim 3, wherein the branch stub further stores an offset tothe original branch instruction and a copy of the original branchinstruction.
 5. The method of claim 4, wherein the branch stub furtherstores a last target address specifying a target address of a lastexecution of the rewritten branch instruction during runtime executionof the code.
 6. The method of claim 2, wherein determining if the targetinstruction of the branch instruction is present in the instructioncache associated with the processor comprises using the branch stub todetermine if a target instruction, corresponding to the original targetaddress of the original branch instruction, is present in theinstruction cache.
 7. The method of claim 6, further comprising:determining, based on the branch stub, if the rewritten branchinstruction can be dynamically rewritten to have a target addresscorresponding to a current location of the original target address inthe instruction cache; and dynamically rewriting the rewritten branchinstruction to target the current location of the original targetaddress in the instruction cache in response to a determination that therewritten branch instruction can be dynamically rewritten.
 8. The methodof claim 1, wherein determining if the target instruction of the branchinstruction is present in the instruction cache associated with theprocessor comprises executing a check instruction prior to the branchinstruction, wherein the check instruction checks whether a currenttarget of the branch instruction is a same target as a previousexecution of the branch instruction.
 9. The method of claim 8, whereinthe check instruction, for a current execution of the branchinstruction, checks a current target address of a current target of thebranch instruction against a last target address specified in the branchstub for a last prior execution of the branch instruction.
 10. Themethod of claim 9, wherein if the last target address and the currenttarget address match, then the processor determines that execution ofthe branch instruction may be branched directly to the current target inthe instruction cache without intervention by the instruction cacheruntime system.
 11. A computer program product comprising a computerreadable storage medium having a computer readable program storedtherein, wherein the computer readable program, when executed on a dataprocessing system, causes the data processing system to: execute abranch instruction in the portion of code; determine if a targetinstruction of the branch instruction, to which the branch instructionbranches, is present in an instruction cache of the data processingsystem; directly branch execution of the portion of code to the targetinstruction in the instruction cache, without intervention from aninstruction cache runtime system, in response to a determination thatthe target instruction is present in the instruction cache; and redirectexecution of the portion of code to the instruction cache runtime systemin response to a determination that the target instruction cannot bedetermined to be present in the instruction cache.
 12. The computerprogram product of claim 11, wherein the branch instruction is arewritten branch instruction that has been rewritten by a compiler froman original branch instruction having an original target address, to arewritten branch instruction with an associated branch stub.
 13. Thecomputer program product of claim 12, wherein the branch stub stores alink to an instruction cache manager of the instruction cache runtimesystem and the original target address of the original branchinstruction.
 14. The computer program product of claim 13, wherein thebranch stub further stores an offset to the original branch instructionand a copy of the original branch instruction.
 15. The computer programproduct of claim 14, wherein the branch stub further stores a lasttarget address specifying a target address of a last execution of therewritten branch instruction during runtime execution of the code. 16.The computer program product of claim 12, wherein determining if thetarget instruction of the branch instruction is present in theinstruction cache associated with the processor comprises using thebranch stub to determine if a target instruction, corresponding to theoriginal target address of the original branch instruction, is presentin the instruction cache.
 17. The computer program product of claim 16,wherein the computer readable program further causes the data processingsystem to: determine, based on the branch stub, if the rewritten branchinstruction can be dynamically rewritten to have a target addresscorresponding to a current location of the original target address inthe instruction cache; and dynamically rewrite the rewritten branchinstruction to target the current location of the original targetaddress in the instruction cache in response to a determination that therewritten branch instruction can be dynamically rewritten.
 18. Thecomputer program product of claim 11, wherein determining if the targetinstruction of the branch instruction is present in the instructioncache associated with the processor comprises executing a checkinstruction prior to the branch instruction, wherein the checkinstruction checks whether a current target of the branch instruction isa same target as a previous execution of the branch instruction.
 19. Thecomputer program product of claim 18, wherein the check instruction, fora current execution of the branch instruction, checks a current targetaddress of a current target of the branch instruction against a lasttarget address specified in the branch stub for a last prior executionof the branch instruction.
 20. The computer program product of claim 19,wherein if the last target address and the current target address match,then the processor determines that execution of the branch instructionmay be branched directly to the current target in the instruction cachewithout intervention by the instruction cache runtime system.
 21. A dataprocessing system, comprising: a processor; and an instruction cachecoupled to the processor, wherein the processor is configured to:execute a branch instruction in the portion of code; determine if atarget instruction of the branch instruction, to which the branchinstruction branches, is present in the instruction cache; directlybranch execution of the portion of code to the target instruction in theinstruction cache, without intervention from an instruction cacheruntime system, in response to a determination that the targetinstruction is present in the instruction cache; and redirect executionof the portion of code to the instruction cache runtime system inresponse to a determination that the target instruction cannot bedetermined to be present in the instruction cache.
 22. The dataprocessing system of claim 21, wherein the branch instruction is arewritten branch instruction that has been rewritten by a compiler froman original branch instruction having an original target address, to arewritten branch instruction with an associated branch stub.
 23. Thedata processing system of claim 22, wherein the branch stub stores alink to an instruction cache manager of the instruction cache runtimesystem and the original target address of the original branchinstruction, and one or more of an offset to the original branchinstruction, a copy of the original branch instruction, or, for indirectbranch instructions, a last target address specifying a target addressof a last execution of the rewritten branch instruction during runtimeexecution of the code.
 24. The data processing system of claim 22,wherein determining if the target instruction of the branch instructionis present in the instruction cache associated with the processorcomprises using the branch stub to determine if a target instruction,corresponding to the original target address of the original branchinstruction, is present in the instruction cache, and wherein theprocessor is further configured to: determine, based on the branch stub,if the rewritten branch instruction can be dynamically rewritten to havea target address corresponding to a current location of the originaltarget address in the instruction cache; and dynamically rewrite therewritten branch instruction to target the current location of theoriginal target address in the instruction cache in response to adetermination that the rewritten branch instruction can be dynamicallyrewritten.
 25. The computer program product of claim 11, wherein:determining if the target instruction of the branch instruction ispresent in the instruction cache associated with the processor comprisesexecuting a check instruction prior to the branch instruction, the checkinstruction checks whether a current target of the branch instruction isa same target as a previous execution of the branch instruction, thecheck instruction, for a current execution of the branch instruction,checks a current target address of a current target of the branchinstruction against a last target address specified in the branch stubfor a last prior execution of the branch instruction, and if the lasttarget address and the current target address match, then the processordetermines that execution of the branch instruction may be brancheddirectly to the current target in the instruction cache withoutintervention by the instruction cache runtime system.